{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8e56e34e",
   "metadata": {},
   "source": [
    "通过本次考试，主要对python编程能力、深度学习基础知识及实践能力进行测试，python要用3.8版本及以上，pytorch要用2.0及以上版本编写代码，允许使用第三方库（如 NumPy、Pandas 等辅助工具）。一共5道题，每题20分。\n",
    "第一题，主要考核关于面向对象的编程能力，涉及类、继承、接口、构造函数等相关概念及编程。\n",
    "第二题，主要考核对感知器、基本激活函数、交叉熵损失函数、逻辑回归的掌握情况，要求python实现。\n",
    "第三题，主要考核利用pytorch2.0实现基于CNN的神经网络模型，需要掌握CNN相关的概念及pytorch的相关知识点。需要按照提供的模型图或描述、参数等编写神经网络的类。\n",
    "第四题，主要考核利用pytorch2.0实现基于LSTM的神经网络模型，需要掌握LSTM相关的概念及pytorch的相关知识点。需要按照提供的模型图或描述、参数等编写神经网络的类。\n",
    "\n",
    "第五题，主要考核利用pytorch2.0实现基于transformer和lstm或cnn综合应用的编程能力。需要按照提供的模型图或描述、参数等编写神经网络的类。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b52d014a",
   "metadata": {},
   "source": [
    "1.第一题，主要考核关于面向对象的编程能力，涉及类、继承、接口、构造函数等相关概念及编程。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33a5e678",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "\n",
    "          \n",
    "\n",
    "\n",
    "\n",
    "## 接口 (Interface)\n",
    "\n",
    "### 定义\n",
    "接口是一种抽象类型，它定义了一组方法和属性，但不提供具体的实现。接口充当了类与外部世界之间的契约，规定了实现该接口的类必须提供哪些方法。\n",
    "\n",
    "### 特点\n",
    "1. **抽象性**：接口只定义方法签名，不包含实现代码\n",
    "2. **契约性**：实现接口的类必须提供接口中定义的所有方法\n",
    "3. **多态性**：通过接口可以实现多态，不同的类可以实现同一个接口但有不同的行为\n",
    "\n",
    "### 在Python中的实现\n",
    "Python没有像Java那样的`interface`关键字，但可以通过以下方式实现接口：\n",
    "\n",
    "1. **抽象基类 (Abstract Base Classes)**：\n",
    "```python\n",
    "from abc import ABC, abstractmethod\n",
    "\n",
    "class Shape(ABC):  # 这是一个接口/抽象基类\n",
    "    @abstractmethod\n",
    "    def area(self):\n",
    "        pass\n",
    "    \n",
    "    @abstractmethod\n",
    "    def perimeter(self):\n",
    "        pass\n",
    "```\n",
    "\n",
    "2. **协议 (Protocol)** - Python 3.8+：\n",
    "```python\n",
    "from typing import Protocol\n",
    "\n",
    "class Drawable(Protocol):\n",
    "    def draw(self) -> None: ...\n",
    "```\n",
    "\n",
    "## 构造函数 (Constructor)\n",
    "\n",
    "### 定义\n",
    "构造函数是一个特殊的方法，在创建对象时自动调用，用于初始化对象的状态。在Python中，构造函数是`__init__`方法。\n",
    "\n",
    "### 特点\n",
    "1. **自动调用**：创建对象时自动执行\n",
    "2. **初始化对象**：设置对象的初始状态和属性值\n",
    "3. **只执行一次**：每个对象的生命周期中只调用一次\n",
    "\n",
    "### 在Python中的实现\n",
    "```python\n",
    "class Person:\n",
    "    def __init__(self, name, age):  # 这是构造函数\n",
    "        self.name = name  # 初始化属性\n",
    "        self.age = age\n",
    "    \n",
    "    def introduce(self):\n",
    "        return f\"我叫{self.name}，今年{self.age}岁\"\n",
    "\n",
    "# 创建对象时自动调用构造函数\n",
    "person = Person(\"张三\", 25)  # __init__方法被调用\n",
    "```\n",
    "\n",
    "## 在第一题中的应用示例\n",
    "\n",
    "让我查看第一题的具体实现，以便更好地解释这两个概念：\n",
    "\n",
    "        \n",
    "            \n",
    "toolName: view_files\n",
    "            \n",
    "status: success\n",
    "          \n",
    "            \n",
    "filePath: d:\\MyFile\\WareHouse\\Gitee\\daily_code\\编程测试\\1.py\n",
    "          \n",
    "\n",
    "现在我可以结合第一题的具体代码来详细解释接口和构造函数：\n",
    "\n",
    "## 接口在第一题中的应用\n",
    "\n",
    "在第一题中，<mcsymbol name=\"Shape\" filename=\"1.py\" path=\"d:\\MyFile\\WareHouse\\Gitee\\daily_code\\编程测试\\1.py\" startline=\"8\" type=\"class\"></mcsymbol> 类就是一个接口（在Python中称为抽象基类）：\n",
    "\n",
    "```python\n",
    "from abc import ABC, abstractmethod\n",
    "\n",
    "class Shape(ABC):\n",
    "    \"\"\"抽象基类，定义图形接口\"\"\"\n",
    "    \n",
    "    @abstractmethod\n",
    "    def area(self):\n",
    "        \"\"\"计算面积的抽象方法\"\"\"\n",
    "        pass\n",
    "    \n",
    "    @abstractmethod\n",
    "    def perimeter(self):\n",
    "        \"\"\"计算周长的抽象方法\"\"\"\n",
    "        pass\n",
    "```\n",
    "\n",
    "这个接口定义了所有图形必须实现的方法：\n",
    "1. <mcsymbol name=\"area\" filename=\"1.py\" path=\"d:\\MyFile\\WareHouse\\Gitee\\daily_code\\编程测试\\1.py\" startline=\"15\" type=\"function\"></mcsymbol> - 计算面积的方法\n",
    "2. <mcsymbol name=\"perimeter\" filename=\"1.py\" path=\"d:\\MyFile\\WareHouse\\Gitee\\daily_code\\编程测试\\1.py\" startline=\"20\" type=\"function\"></mcsymbol> - 计算周长的方法\n",
    "\n",
    "任何继承自Shape的类（如<mcsymbol name=\"Circle\" filename=\"1.py\" path=\"d:\\MyFile\\WareHouse\\Gitee\\daily_code\\编程测试\\1.py\" startline=\"25\" type=\"class\"></mcsymbol>、<mcsymbol name=\"Rectangle\" filename=\"1.py\" path=\"d:\\MyFile\\WareHouse\\Gitee\\daily_code\\编程测试\\1.py\" startline=\"44\" type=\"class\"></mcsymbol>、<mcsymbol name=\"Square\" filename=\"1.py\" path=\"d:\\MyFile\\WareHouse\\Gitee\\daily_code\\编程测试\\1.py\" startline=\"66\" type=\"class\"></mcsymbol>）都必须实现这两个方法，这就是接口的契约性。\n",
    "\n",
    "## 构造函数在第一题中的应用\n",
    "\n",
    "在第一题中，每个类都有自己的构造函数<mcsymbol name=\"__init__\" filename=\"1.py\" path=\"d:\\MyFile\\WareHouse\\Gitee\\daily_code\\编程测试\\1.py\" startline=\"10\" type=\"function\"></mcsymbol>：\n",
    "\n",
    "1. **Shape类的构造函数**：\n",
    "```python\n",
    "def __init__(self, color):\n",
    "    \"\"\"构造函数，初始化颜色\"\"\"\n",
    "    self.color = color\n",
    "```\n",
    "\n",
    "2. **Circle类的构造函数**：\n",
    "```python\n",
    "def __init__(self, color, radius):\n",
    "    \"\"\"构造函数，调用父类构造函数并初始化半径\"\"\"\n",
    "    super().__init__(color)  # 调用父类构造函数\n",
    "    self.radius = radius     # 初始化子类特有属性\n",
    "```\n",
    "\n",
    "3. **Rectangle类的构造函数**：\n",
    "```python\n",
    "def __init__(self, color, width, height):\n",
    "    \"\"\"构造函数，调用父类构造函数并初始化宽和高\"\"\"\n",
    "    super().__init__(color)  # 调用父类构造函数\n",
    "    self.width = width       # 初始化子类特有属性\n",
    "    self.height = height     # 初始化子类特有属性\n",
    "```\n",
    "\n",
    "4. **Square类的构造函数**：\n",
    "```python\n",
    "def __init__(self, color, side):\n",
    "    \"\"\"构造函数，调用父类构造函数并初始化边长\"\"\"\n",
    "    super().__init__(color, side, side)  # 调用父类构造函数\n",
    "    self.side = side                     # 初始化子类特有属性\n",
    "```\n",
    "\n",
    "## 关键概念总结\n",
    "\n",
    "1. **接口**：\n",
    "   - 定义了一组方法签名，但不提供实现\n",
    "   - 在Python中通过抽象基类(ABC)和@abstractmethod装饰器实现\n",
    "   - 提供了一种规范，确保所有子类都实现相同的方法\n",
    "\n",
    "2. **构造函数**：\n",
    "   - 在Python中是`__init__`方法\n",
    "   - 用于初始化对象的属性和状态\n",
    "   - 在子类中可以使用`super()`调用父类的构造函数\n",
    "\n",
    "3. **继承关系**：\n",
    "   - Circle和Rectangle继承自Shape接口\n",
    "   - Square继承自Rectangle（多层继承）\n",
    "   - 每个子类都必须实现接口中定义的抽象方法\n",
    "\n",
    "4. **多态性**：\n",
    "   - 不同的类可以实现相同的接口方法但有不同的行为\n",
    "   - 例如，Circle和Rectangle都有area()方法，但计算方式不同\n",
    "\n",
    "这些概念共同构成了面向对象编程的基础，使得代码更加模块化、可扩展和易于维护。\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d5377bf",
   "metadata": {},
   "outputs": [],
   "source": [
    "from abc import ABC, abstractmethod\n",
    "import datetime\n",
    "\n",
    "# 定义一个接口（抽象基类），规定所有动物必须实现的方法\n",
    "class Animal(ABC):\n",
    "    def __init__(self, name, age, species):\n",
    "        self.name = name\n",
    "        self.age = age\n",
    "        self.species = species\n",
    "        self.admission_date = datetime.date.today()\n",
    "        self.is_adopted = False\n",
    "\n",
    "    @abstractmethod\n",
    "    def make_sound(self):\n",
    "        \"\"\"抽象方法，子类必须实现\"\"\"\n",
    "        pass\n",
    "\n",
    "    def get_info(self):\n",
    "        status = \"已领养\" if self.is_adopted else \"待领养\"\n",
    "        return f\"名字: {self.name}, 年龄: {self.age}, 种类: {self.species}, 入所日期: {self.admission_date}, 状态: {status}\"\n",
    "\n",
    "    def adopt(self):\n",
    "        self.is_adopted = True\n",
    "        print(f\"{self.name} 已被成功领养！\")\n",
    "\n",
    "# Dog 类继承 Animal 类\n",
    "class Dog(Animal):\n",
    "    def __init__(self, name, age, breed):\n",
    "        super().__init__(name, age, \"狗\")  # 调用父类构造函数\n",
    "        self.breed = breed  # Dog 类特有的属性\n",
    "\n",
    "    def make_sound(self):\n",
    "        return f\"{self.name} 汪汪叫！\"\n",
    "\n",
    "    def get_info(self):\n",
    "        # 重写父类方法，添加品种信息\n",
    "        base_info = super().get_info()\n",
    "        return f\"{base_info}, 品种: {self.breed}\"\n",
    "\n",
    "# Cat 类继承 Animal 类\n",
    "class Cat(Animal):\n",
    "    def __init__(self, name, age, is_indoor=True):\n",
    "        super().__init__(name, age, \"猫\")  # 调用父类构造函数\n",
    "        self.is_indoor = is_indoor  # Cat 类特有的属性\n",
    "\n",
    "    def make_sound(self):\n",
    "        return f\"{self.name} 喵喵叫！\"\n",
    "\n",
    "    def get_info(self):\n",
    "        # 重写父类方法，添加室内猫信息\n",
    "        base_info = super().get_info()\n",
    "        indoor_status = \"室内猫\" if self.is_indoor else \"室外猫\"\n",
    "        return f\"{base_info}, 类型: {indoor_status}\"\n",
    "\n",
    "# Shelter 类管理动物\n",
    "class Shelter:\n",
    "    def __init__(self, name):\n",
    "        self.name = name\n",
    "        self.animals = []\n",
    "\n",
    "    def add_animal(self, animal: Animal):\n",
    "        self.animals.append(animal)\n",
    "        print(f\"{animal.name} 已被添加到 {self.name}。\")\n",
    "\n",
    "    def list_animals(self):\n",
    "        print(f\"\\n--- {self.name} 的动物列表 ---\")\n",
    "        for animal in self.animals:\n",
    "            print(animal.get_info())\n",
    "\n",
    "    def find_animal_by_name(self, name):\n",
    "        for animal in self.animals:\n",
    "            if animal.name == name:\n",
    "                return animal\n",
    "        return None\n",
    "\n",
    "# 演示用法\n",
    "if __name__ == \"__main__\":\n",
    "    shelter = Shelter(\"阳光宠物收容所\")\n",
    "\n",
    "    # 创建 Dog 和 Cat 对象，展示了构造函数的使用\n",
    "    dog1 = Dog(\"旺财\", 3, \"金毛\")\n",
    "    cat1 = Cat(\"咪咪\", 2, is_indoor=True)\n",
    "\n",
    "    # 添加动物到收容所\n",
    "    shelter.add_animal(dog1)\n",
    "    shelter.add_animal(cat1)\n",
    "\n",
    "    # 展示多态性：调用不同子类的 make_sound 方法\n",
    "    print(\"\\n--- 动物叫声 ---\")\n",
    "    for animal in shelter.animals:\n",
    "        print(animal.make_sound()) # 通过接口调用，体现多态\n",
    "\n",
    "    # 列出所有动物信息\n",
    "    shelter.list_animals()\n",
    "\n",
    "    # 模拟领养\n",
    "    dog1.adopt()\n",
    "    print(f\"\\n--- {dog1.name} 领养后信息 ---\")\n",
    "    print(dog1.get_info())\n",
    "\n",
    "    # 查找动物\n",
    "    found_cat = shelter.find_animal_by_name(\"咪咪\")\n",
    "    if found_cat:\n",
    "        print(f\"\\n找到动物: {found_cat.get_info()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d0d36cb",
   "metadata": {},
   "source": [
    "2、第二题，主要考核对感知器、基本激活函数、交叉熵损失函数、逻辑回归的掌握情况，要求python实现。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4295ffc6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "# 1. 感知器 (Perceptron)\n",
    "class Perceptron:\n",
    "    def __init__(self, learning_rate=0.01, n_iters=1000):\n",
    "        self.lr = learning_rate\n",
    "        self.n_iters = n_iters\n",
    "        self.weights = None\n",
    "        self.bias = None\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        n_samples, n_features = X.shape\n",
    "        \n",
    "        # 初始化权重和偏置\n",
    "        self.weights = np.zeros(n_features)#每个权重对应一个特征\n",
    "        self.bias = 0\n",
    "        \n",
    "        # 感知器学习算法\n",
    "        for _ in range(self.n_iters):\n",
    "            for idx, x_i in enumerate(X):\n",
    "                # 线性组合\n",
    "                linear_output = np.dot(x_i, self.weights) + self.bias\n",
    "                # 预测（使用单位阶跃函数）\n",
    "                y_predicted = self._unit_step_function(linear_output)\n",
    "\n",
    "                # 更新权重和偏置\n",
    "                update = self.lr * (y[idx] - y_predicted)\n",
    "                self.weights += update * x_i\n",
    "                self.bias += update\n",
    "\n",
    "    def _unit_step_function(self, x):#单位阶跃\n",
    "        return np.where(x >= 0, 1, 0)#如果输入值 x 大于或等于0，返回1\n",
    "\n",
    "    def predict(self, X):\n",
    "        linear_output = np.dot(X, self.weights) + self.bias\n",
    "        y_predicted = self._unit_step_function(linear_output)\n",
    "        return y_predicted\n",
    "\n",
    "# 2. 激活函数\n",
    "def sigmoid(x):\n",
    "    \"\"\"Sigmoid 激活函数\"\"\"\n",
    "    # 防止溢出\n",
    "    x = np.clip(x, -500, 500)\n",
    "    return 1 / (1 + np.exp(-x))\n",
    "\n",
    "def relu(x):\n",
    "    \"\"\"ReLU 激活函数\"\"\"\n",
    "    return np.maximum(0, x)\n",
    "\n",
    "def tanh(x):\n",
    "    \"\"\"Tanh 激活函数\"\"\"\n",
    "    return np.tanh(x)\n",
    "\n",
    "# 3. 交叉熵损失函数\n",
    "def cross_entropy_loss(y_true, y_pred_proba):\n",
    "    \"\"\"\n",
    "    计算二分类交叉熵损失\n",
    "    y_true: 真实标签 (0 或 1)\n",
    "    y_pred_proba: 预测为正类的概率\n",
    "    \"\"\"\n",
    "    # 防止 log(0) 导致无穷大\n",
    "    epsilon = 1e-15\n",
    "    y_pred_proba = np.clip(y_pred_proba, epsilon, 1 - epsilon)\n",
    "    \n",
    "    loss = - (y_true * np.log(y_pred_proba) + (1 - y_true) * np.log(1 - y_pred_proba))\n",
    "    return np.mean(loss)\n",
    "\n",
    "def cross_entropy_loss_multiclass(y_true, y_pred_proba):\n",
    "    \"\"\"\n",
    "    计算多分类交叉熵损失\n",
    "    y_true: 真实标签 (整数类别索引) 或 one-hot 编码\n",
    "    y_pred_proba: 预测概率矩阵 (样本数 x 类别数)\n",
    "    \"\"\"\n",
    "    if y_true.ndim == 1:\n",
    "        # 如果 y_true 是类别索引，转换为 one-hot\n",
    "        num_classes = y_pred_proba.shape[1]\n",
    "        y_true_onehot = np.eye(num_classes)[y_true]\n",
    "    else:\n",
    "        y_true_onehot = y_true\n",
    "    \n",
    "    # 防止 log(0)\n",
    "    epsilon = 1e-15\n",
    "    y_pred_proba = np.clip(y_pred_proba, epsilon, 1 - epsilon)\n",
    "    \n",
    "    loss = -np.sum(y_true_onehot * np.log(y_pred_proba)) / y_pred_proba.shape[0]\n",
    "    return loss\n",
    "\n",
    "# 4. 逻辑回归 (Logistic Regression)\n",
    "class LogisticRegression:\n",
    "    def __init__(self, learning_rate=0.001, n_iters=1000):\n",
    "        self.lr = learning_rate\n",
    "        self.n_iters = n_iters\n",
    "        self.weights = None\n",
    "        self.bias = None\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        n_samples, n_features = X.shape\n",
    "        \n",
    "        # 初始化参数\n",
    "        self.weights = np.zeros(n_features)\n",
    "        self.bias = 0\n",
    "        \n",
    "        # 梯度下降\n",
    "        for i in range(self.n_iters):\n",
    "            # 线性预测\n",
    "            linear_pred = np.dot(X, self.weights) + self.bias\n",
    "            # 应用 Sigmoid 激活函数\n",
    "            predictions = sigmoid(linear_pred)\n",
    "            \n",
    "            # 计算梯度\n",
    "            dw = (1 / n_samples) * np.dot(X.T, (predictions - y))\n",
    "            db = (1 / n_samples) * np.sum(predictions - y)\n",
    "            \n",
    "            # 更新参数\n",
    "            self.weights -= self.lr * dw\n",
    "            self.bias -= self.lr * db\n",
    "            \n",
    "            # (可选) 计算并打印损失\n",
    "            if i % 200 == 0:\n",
    "                loss = cross_entropy_loss(y, predictions)\n",
    "                print(f\"Iteration {i}, Loss: {loss:.4f}\")\n",
    "\n",
    "    def predict_proba(self, X):\n",
    "        \"\"\"返回预测为正类的概率\"\"\"\n",
    "        linear_pred = np.dot(X, self.weights) + self.bias\n",
    "        return sigmoid(linear_pred)\n",
    "\n",
    "    def predict(self, X):\n",
    "        \"\"\"返回二分类预测结果\"\"\"\n",
    "        predictions = self.predict_proba(X)\n",
    "        return [1 if i > 0.5 else 0 for i in predictions]\n",
    "\n",
    "# 演示用法\n",
    "if __name__ == \"__main__\":\n",
    "    print(\"--- 1. 感知器演示 ---\")\n",
    "    # 感知器数据 (简单的 AND 门)\n",
    "    X_and = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])\n",
    "    y_and = np.array([0, 0, 0, 1])\n",
    "    \n",
    "    p = Perceptron(learning_rate=0.1, n_iters=1000)\n",
    "    p.fit(X_and, y_and)\n",
    "    predictions_perceptron = p.predict(X_and)\n",
    "    print(f\"感知器预测结果: {predictions_perceptron}\")\n",
    "    print(f\"真实标签:       {y_and}\")\n",
    "    print(f\"感知器权重: {p.weights}, 偏置: {p.bias}\\n\")\n",
    "\n",
    "    print(\"--- 2. 激活函数演示 ---\")\n",
    "    test_input = np.array([-1, 0, 1, 2])\n",
    "    print(f\"输入: {test_input}\")\n",
    "    print(f\"Sigmoid 输出: {sigmoid(test_input)}\")\n",
    "    print(f\"ReLU 输出: {relu(test_input)}\")\n",
    "    print(f\"Tanh 输出: {tanh(test_input)}\\n\")\n",
    "\n",
    "    print(\"--- 3. 交叉熵损失演示 ---\")\n",
    "    y_true_binary = np.array([1, 0, 1, 1])\n",
    "    y_pred_proba_binary = np.array([0.9, 0.1, 0.8, 0.6])\n",
    "    loss_binary = cross_entropy_loss(y_true_binary, y_pred_proba_binary)\n",
    "    print(f\"二分类交叉熵损失: {loss_binary:.4f}\\n\")\n",
    "\n",
    "    print(\"--- 4. 逻辑回归演示 ---\")\n",
    "    # 生成一些示例数据\n",
    "    np.random.seed(42)\n",
    "    X_logreg = np.random.randn(100, 2)\n",
    "    y_logreg = (X_logreg[:, 0] + X_logreg[:, 1] > 0).astype(int)\n",
    "    \n",
    "    logreg = LogisticRegression(learning_rate=0.1, n_iters=1000)\n",
    "    logreg.fit(X_logreg, y_logreg)\n",
    "    \n",
    "    # 测试预测\n",
    "    test_point = np.array([[0.5, 0.5]])\n",
    "    proba = logreg.predict_proba(test_point)\n",
    "    pred = logreg.predict(test_point)\n",
    "    print(f\"测试点 {test_point[0]} 的预测概率: {proba[0]:.4f}\")\n",
    "    print(f\"测试点 {test_point[0]} 的预测类别: {pred[0]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4854f84a",
   "metadata": {},
   "source": [
    "3.第三题，主要考核利用pytorch2.0实现基于CNN的神经网络模型，需要掌握CNN相关的概念及pytorch的相关知识点。需要按照提供的模型图或描述、参数等编写神经网络的类。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b10c964",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "from torch.utils.data import DataLoader\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms\n",
    "\n",
    "# 定义一个简单的CNN模型\n",
    "class SimpleCNN(nn.Module):\n",
    "    def __init__(self, num_classes=10):\n",
    "        super(SimpleCNN, self).__init__()\n",
    "        \n",
    "        # 第一个卷积块\n",
    "        self.conv1 = nn.Conv2d(in_channels=3, out_channels=32, kernel_size=3, padding=1)  # 输出 32x32x32\n",
    "        self.bn1 = nn.BatchNorm2d(32)\n",
    "        self.pool1 = nn.MaxPool2d(kernel_size=2, stride=2)  # 输出 16x16x32\n",
    "        self.dropout1 = nn.Dropout2d(0.25)\n",
    "        \n",
    "        # 第二个卷积块\n",
    "        self.conv2 = nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3, padding=1) # 输出 16x16x64\n",
    "        self.bn2 = nn.BatchNorm2d(64)\n",
    "        self.pool2 = nn.MaxPool2d(kernel_size=2, stride=2)  # 输出 8x8x64\n",
    "        self.dropout2 = nn.Dropout2d(0.25)\n",
    "        \n",
    "        # 第三个卷积块\n",
    "        self.conv3 = nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, padding=1) # 输出 8x8x128\n",
    "        self.bn3 = nn.BatchNorm2d(128)\n",
    "        self.pool3 = nn.MaxPool2d(kernel_size=2, stride=2)  # 输出 4x4x128\n",
    "        self.dropout3 = nn.Dropout2d(0.25)\n",
    "        \n",
    "        # 全连接层\n",
    "        # 经过三次池化后，如果输入是 32x32，则特征图大小为 4x4\n",
    "        # 最后一个卷积层输出128个特征图，所以展平后是 4*4*128 = 2048\n",
    "        self.fc1 = nn.Linear(4 * 4 * 128, 512)\n",
    "        self.dropout_fc1 = nn.Dropout(0.5)\n",
    "        self.fc2 = nn.Linear(512, num_classes)\n",
    "\n",
    "    def forward(self, x):\n",
    "        # 第一个卷积块\n",
    "        x = self.pool1(F.relu(self.bn1(self.conv1(x))))\n",
    "        x = self.dropout1(x)\n",
    "        \n",
    "        # 第二个卷积块\n",
    "        x = self.pool2(F.relu(self.bn2(self.conv2(x))))\n",
    "        x = self.dropout2(x)\n",
    "        \n",
    "        # 第三个卷积块\n",
    "        x = self.pool3(F.relu(self.bn3(self.conv3(x))))\n",
    "        x = self.dropout3(x)\n",
    "        \n",
    "        # 展平\n",
    "        x = x.view(x.size(0), -1)  # (batch_size, 4*4*128)\n",
    "        \n",
    "        # 全连接层\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = self.dropout_fc1(x)\n",
    "        x = self.fc2(x)  # 输出层不使用激活函数，因为CrossEntropyLoss内部会处理\n",
    "        \n",
    "        return x\n",
    "\n",
    "# 示例：使用CIFAR-10数据集进行训练\n",
    "def train_model():\n",
    "    # 数据预处理\n",
    "    transform = transforms.Compose([\n",
    "        transforms.ToTensor(),\n",
    "        transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)), # CIFAR-10的均值和标准差\n",
    "    ])\n",
    "\n",
    "    # 加载数据集\n",
    "    trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)\n",
    "    trainloader = DataLoader(trainset, batch_size=64, shuffle=True, num_workers=2)\n",
    "\n",
    "    testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform)\n",
    "    testloader = DataLoader(testset, batch_size=64, shuffle=False, num_workers=2)\n",
    "\n",
    "    # 实例化模型\n",
    "    net = SimpleCNN(num_classes=10)\n",
    "\n",
    "    # 定义损失函数和优化器\n",
    "    criterion = nn.CrossEntropyLoss()\n",
    "    optimizer = optim.Adam(net.parameters(), lr=0.001)\n",
    "\n",
    "    # 训练循环\n",
    "    print(\"开始训练模型...\")\n",
    "    for epoch in range(2):  # 训练2个epoch作为演示\n",
    "        running_loss = 0.0\n",
    "        for i, data in enumerate(trainloader, 0):\n",
    "            inputs, labels = data\n",
    "\n",
    "            # 清零梯度\n",
    "            optimizer.zero_grad()\n",
    "\n",
    "            # 前向传播\n",
    "            outputs = net(inputs)\n",
    "            loss = criterion(outputs, labels)\n",
    "\n",
    "            # 反向传播和优化\n",
    "            loss.backward()\n",
    "            optimizer.step()\n",
    "\n",
    "            running_loss += loss.item()\n",
    "            if i % 100 == 99:    # 每100个小批量打印一次\n",
    "                print(f'Epoch {epoch + 1}, Batch {i + 1}, Loss: {running_loss / 100:.4f}')\n",
    "                running_loss = 0.0\n",
    "\n",
    "    print('训练完成')\n",
    "\n",
    "    # 测试模型\n",
    "    correct = 0\n",
    "    total = 0\n",
    "    with torch.no_grad():\n",
    "        for data in testloader:\n",
    "            images, labels = data\n",
    "            outputs = net(images)\n",
    "            _, predicted = torch.max(outputs.data, 1)\n",
    "            total += labels.size(0)\n",
    "            correct += (predicted == labels).sum().item()\n",
    "\n",
    "    print(f'测试集准确率: {100 * correct / total:.2f}%')\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    # 检查是否有GPU\n",
    "    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "    print(f\"使用设备: {device}\")\n",
    "    \n",
    "    # 创建模型实例并打印结构\n",
    "    model = SimpleCNN(num_classes=10)\n",
    "    print(\"\\n模型结构:\")\n",
    "    print(model)\n",
    "    \n",
    "    # 计算参数量\n",
    "    pytorch_total_params = sum(p.numel() for p in model.parameters())\n",
    "    print(f\"\\n模型总参数量: {pytorch_total_params}\")\n",
    "    \n",
    "    # 运行训练示例 (注意：实际运行可能需要下载数据集)\n",
    "    # train_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e72c6b09",
   "metadata": {},
   "source": [
    "4.第四题，主要考核利用pytorch2.0实现基于LSTM的神经网络模型，需要掌握LSTM相关的概念及pytorch的相关知识点。需要按照提供的模型图或描述、参数等编写神经网络的类。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a70495fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "import numpy as np\n",
    "\n",
    "# LSTM模型定义\n",
    "class LSTMTextClassifier(nn.Module):\n",
    "    def __init__(self, vocab_size, embedding_dim, hidden_dim, num_layers, output_dim, dropout=0.3):\n",
    "        super(LSTMTextClassifier, self).__init__()\n",
    "        \n",
    "        self.hidden_dim = hidden_dim\n",
    "        self.num_layers = num_layers\n",
    "        \n",
    "        # 词嵌入层\n",
    "        self.embedding = nn.Embedding(vocab_size, embedding_dim)\n",
    "        \n",
    "        # LSTM层\n",
    "        self.lstm = nn.LSTM(embedding_dim, hidden_dim, num_layers, \n",
    "                            batch_first=True, dropout=dropout if num_layers > 1 else 0)\n",
    "        \n",
    "        # Dropout层\n",
    "        self.dropout = nn.Dropout(dropout)\n",
    "        \n",
    "        # 全连接输出层\n",
    "        self.fc = nn.Linear(hidden_dim, output_dim)\n",
    "        \n",
    "        # Sigmoid激活函数用于二分类\n",
    "        self.sigmoid = nn.Sigmoid()\n",
    "\n",
    "    def forward(self, x):\n",
    "        # x shape: (batch_size, sequence_length)\n",
    "        embedded = self.embedding(x)  # (batch_size, seq_len, embedding_dim)\n",
    "        \n",
    "        # 初始化隐藏状态和细胞状态\n",
    "        h0 = torch.zeros(self.num_layers, x.size(0), self.hidden_dim).to(x.device)\n",
    "        c0 = torch.zeros(self.num_layers, x.size(0), self.hidden_dim).to(x.device)\n",
    "        \n",
    "        # LSTM前向传播\n",
    "        lstm_out, (hn, cn) = self.lstm(embedded, (h0, c0))\n",
    "        \n",
    "        # 取最后一个时间步的输出\n",
    "        last_output = lstm_out[:, -1, :]  # (batch_size, hidden_dim)\n",
    "        \n",
    "        # 应用dropout\n",
    "        last_output = self.dropout(last_output)\n",
    "        \n",
    "        # 全连接层\n",
    "        output = self.fc(last_output)  # (batch_size, output_dim)\n",
    "        \n",
    "        # 应用Sigmoid激活\n",
    "        output = self.sigmoid(output)\n",
    "        \n",
    "        return output\n",
    "\n",
    "# 模拟数据生成器\n",
    "def generate_sample_data(batch_size=32, seq_length=20, vocab_size=1000, num_classes=2):\n",
    "    \"\"\"\n",
    "    生成模拟的文本分类数据\n",
    "    \"\"\"\n",
    "    # 生成随机的文本序列 (batch_size, seq_length)\n",
    "    text_sequences = torch.randint(1, vocab_size, (batch_size, seq_length))\n",
    "    \n",
    "    # 生成随机标签 (batch_size,)\n",
    "    labels = torch.randint(0, num_classes, (batch_size,))\n",
    "    \n",
    "    return text_sequences, labels\n",
    "\n",
    "# 训练函数\n",
    "def train_lstm_model():\n",
    "    # 模型参数\n",
    "    vocab_size = 10000  # 词汇表大小\n",
    "    embedding_dim = 128  # 词嵌入维度\n",
    "    hidden_dim = 256     # LSTM隐藏层维度\n",
    "    num_layers = 2       # LSTM层数\n",
    "    output_dim = 1       # 输出维度 (二分类)\n",
    "    num_epochs = 5\n",
    "    learning_rate = 0.001\n",
    "    \n",
    "    # 实例化模型\n",
    "    model = LSTMTextClassifier(vocab_size, embedding_dim, hidden_dim, num_layers, output_dim)\n",
    "    \n",
    "    # 定义损失函数和优化器\n",
    "    criterion = nn.BCELoss()  # 二分类交叉熵损失\n",
    "    optimizer = optim.Adam(model.parameters(), lr=learning_rate)\n",
    "    \n",
    "    print(\"开始训练LSTM模型...\")\n",
    "    \n",
    "    for epoch in range(num_epochs):\n",
    "        total_loss = 0.0\n",
    "        \n",
    "        # 模拟多个批次\n",
    "        for batch_idx in range(100):  # 每个epoch有100个批次\n",
    "            # 生成批次数据\n",
    "            inputs, targets = generate_sample_data(batch_size=32, seq_length=20, \n",
    "                                                  vocab_size=vocab_size, num_classes=2)\n",
    "            targets = targets.float().unsqueeze(1)  # 调整标签形状为 (batch_size, 1)\n",
    "            \n",
    "            # 前向传播\n",
    "            outputs = model(inputs)\n",
    "            loss = criterion(outputs, targets)\n",
    "            \n",
    "            # 反向传播和优化\n",
    "            optimizer.zero_grad()\n",
    "            loss.backward()\n",
    "            optimizer.step()\n",
    "            \n",
    "            total_loss += loss.item()\n",
    "        \n",
    "        avg_loss = total_loss / 100\n",
    "        print(f'Epoch [{epoch+1}/{num_epochs}], Average Loss: {avg_loss:.4f}')\n",
    "    \n",
    "    print(\"训练完成!\")\n",
    "    \n",
    "    return model\n",
    "\n",
    "# 模型结构说明\n",
    "def print_model_info():\n",
    "    print(\"LSTM模型结构说明:\")\n",
    "    print(\"- 词嵌入层: 将词汇索引转换为密集向量表示\")\n",
    "    print(\"- LSTM层: 包含2层，隐藏维度为256，用于捕捉序列信息\")\n",
    "    print(\"- Dropout层: 防止过拟合，丢弃率为0.3\")\n",
    "    print(\"- 全连接层: 将LSTM输出映射到分类标签\")\n",
    "    print(\"- 激活函数: Sigmoid用于二分类输出\")\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    # 检查设备\n",
    "    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "    print(f\"使用设备: {device}\")\n",
    "    \n",
    "    # 打印模型信息\n",
    "    print_model_info()\n",
    "    \n",
    "    # 创建模型实例并打印结构\n",
    "    model = LSTMTextClassifier(vocab_size=10000, \n",
    "                              embedding_dim=128, \n",
    "                              hidden_dim=256, \n",
    "                              num_layers=2, \n",
    "                              output_dim=1)\n",
    "    print(\"\\n模型结构:\")\n",
    "    print(model)\n",
    "    \n",
    "    # 计算参数量\n",
    "    pytorch_total_params = sum(p.numel() for p in model.parameters())\n",
    "    print(f\"\\n模型总参数量: {pytorch_total_params}\")\n",
    "    \n",
    "    # 计算可训练参数量\n",
    "    trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)\n",
    "    print(f\"可训练参数量: {trainable_params}\")\n",
    "    \n",
    "    # 运行训练示例\n",
    "    trained_model = train_lstm_model()\n",
    "    \n",
    "    # 演示单次预测\n",
    "    print(\"\\n演示预测:\")\n",
    "    sample_input = torch.randint(1, 10000, (1, 20))  # (batch_size=1, seq_length=20)\n",
    "    with torch.no_grad():\n",
    "        prediction = trained_model(sample_input)\n",
    "        print(f\"输入序列形状: {sample_input.shape}\")\n",
    "        print(f\"预测输出: {prediction.item():.4f} (接近0表示类别0，接近1表示类别1)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15b23720",
   "metadata": {},
   "source": [
    "5.第五题，主要考核利用pytorch2.0实现基于transformer和lstm或cnn综合应用的编程能力。需要按照提供的模型图或描述、参数等编写神经网络的类。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5dd598c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import math\n",
    "\n",
    "# 位置编码层\n",
    "class PositionalEncoding(nn.Module):\n",
    "    def __init__(self, d_model, max_len=5000):\n",
    "        super(PositionalEncoding, self).__init__()\n",
    "        \n",
    "        pe = torch.zeros(max_len, d_model)\n",
    "        position = torch.arange(0, max_len).unsqueeze(1).float()\n",
    "        \n",
    "        div_term = torch.exp(torch.arange(0, d_model, 2).float() *\n",
    "                             -(math.log(10000.0) / d_model))\n",
    "        \n",
    "        pe[:, 0::2] = torch.sin(position * div_term)\n",
    "        pe[:, 1::2] = torch.cos(position * div_term)\n",
    "        \n",
    "        pe = pe.unsqueeze(0)\n",
    "        self.register_buffer('pe', pe)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        # x shape: (batch_size, seq_len, d_model)\n",
    "        return x + self.pe[:, :x.size(1)]\n",
    "\n",
    "# Transformer编码器层\n",
    "class TransformerEncoderLayer(nn.Module):\n",
    "    def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1):\n",
    "        super(TransformerEncoderLayer, self).__init__()\n",
    "        self.self_attn = nn.MultiheadAttention(d_model, nhead, dropout=dropout)\n",
    "        self.linear1 = nn.Linear(d_model, dim_feedforward)\n",
    "        self.dropout = nn.Dropout(dropout)\n",
    "        self.linear2 = nn.Linear(dim_feedforward, d_model)\n",
    "        \n",
    "        self.norm1 = nn.LayerNorm(d_model)\n",
    "        self.norm2 = nn.LayerNorm(d_model)\n",
    "        self.dropout1 = nn.Dropout(dropout)\n",
    "        self.dropout2 = nn.Dropout(dropout)\n",
    "\n",
    "    def forward(self, src):\n",
    "        # Multi-head attention\n",
    "        attn_output, _ = self.self_attn(src, src, src)\n",
    "        src = src + self.dropout1(attn_output)\n",
    "        src = self.norm1(src)\n",
    "        \n",
    "        # Feed forward network\n",
    "        ff_output = self.linear2(self.dropout(F.relu(self.linear1(src))))\n",
    "        src = src + self.dropout2(ff_output)\n",
    "        src = self.norm2(src)\n",
    "        return src\n",
    "\n",
    "# CNN特征提取器\n",
    "class CNNFeatureExtractor(nn.Module):\n",
    "    def __init__(self, input_channels=3, output_dim=512):\n",
    "        super(CNNFeatureExtractor, self).__init__()\n",
    "        \n",
    "        self.conv_layers = nn.Sequential(\n",
    "            nn.Conv2d(input_channels, 32, kernel_size=3, padding=1),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "            nn.Conv2d(32, 64, kernel_size=3, padding=1),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "            nn.Conv2d(64, 128, kernel_size=3, padding=1),\n",
    "            nn.ReLU(),\n",
    "            nn.AdaptiveAvgPool2d((4, 4))  # 确保输出为固定大小\n",
    "        )\n",
    "        \n",
    "        self.flatten = nn.Flatten()\n",
    "        self.fc = nn.Linear(128 * 4 * 4, output_dim)\n",
    "        self.dropout = nn.Dropout(0.5)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.conv_layers(x)\n",
    "        x = self.flatten(x)\n",
    "        x = self.fc(x)\n",
    "        x = self.dropout(x)\n",
    "        return x\n",
    "\n",
    "# 综合模型：CNN + Transformer\n",
    "class CNNTransformerModel(nn.Module):\n",
    "    def __init__(self, num_classes=10, d_model=512, nhead=8, num_layers=6, \n",
    "                 cnn_output_dim=512, max_seq_len=100):\n",
    "        super(CNNTransformerModel, self).__init__()\n",
    "        \n",
    "        # CNN特征提取器\n",
    "        self.cnn_extractor = CNNFeatureExtractor(output_dim=cnn_output_dim)\n",
    "        \n",
    "        # 将CNN输出映射到Transformer的d_model维度\n",
    "        self.cnn_to_transformer = nn.Linear(cnn_output_dim, d_model)\n",
    "        \n",
    "        # 位置编码\n",
    "        self.pos_encoder = PositionalEncoding(d_model, max_seq_len)\n",
    "        \n",
    "        # Transformer编码器层\n",
    "        self.transformer_layers = nn.ModuleList([\n",
    "            TransformerEncoderLayer(d_model, nhead) \n",
    "            for _ in range(num_layers)\n",
    "        ])\n",
    "        \n",
    "        # 分类头\n",
    "        self.classifier = nn.Sequential(\n",
    "            nn.Linear(d_model, d_model // 2),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(0.5),\n",
    "            nn.Linear(d_model // 2, num_classes)\n",
    "        )\n",
    "        \n",
    "        self.d_model = d_model\n",
    "        self.num_classes = num_classes\n",
    "\n",
    "    def forward(self, images):\n",
    "        # 图像输入 shape: (batch_size, channels, height, width)\n",
    "        \n",
    "        # 1. 使用CNN提取图像特征\n",
    "        cnn_features = self.cnn_extractor(images)  # (batch_size, cnn_output_dim)\n",
    "        \n",
    "        # 2. 将CNN特征映射到Transformer的维度\n",
    "        features = self.cnn_to_transformer(cnn_features)  # (batch_size, d_model)\n",
    "        \n",
    "        # 3. 将特征重塑为序列格式 (batch_size, seq_len=1, d_model)\n",
    "        features = features.unsqueeze(1)  # (batch_size, 1, d_model)\n",
    "        \n",
    "        # 4. 添加位置编码\n",
    "        features = self.pos_encoder(features)\n",
    "        \n",
    "        # 5. 通过Transformer编码器层\n",
    "        output = features\n",
    "        for layer in self.transformer_layers:\n",
    "            output = layer(output)\n",
    "        \n",
    "        # 6. 提取最后一个位置的输出 (即原图像特征的高级表示)\n",
    "        final_output = output[:, 0, :]  # (batch_size, d_model)\n",
    "        \n",
    "        # 7. 通过分类头得到最终预测\n",
    "        logits = self.classifier(final_output)  # (batch_size, num_classes)\n",
    "        \n",
    "        return logits\n",
    "\n",
    "# 替代方案：处理序列数据的CNN-Transformer模型\n",
    "class SequenceCNNTransformerModel(nn.Module):\n",
    "    def __init__(self, vocab_size, embedding_dim=128, num_classes=2, \n",
    "                 d_model=512, nhead=8, num_layers=6, max_seq_len=100):\n",
    "        super(SequenceCNNTransformerModel, self).__init__()\n",
    "        \n",
    "        # 词嵌入层\n",
    "        self.embedding = nn.Embedding(vocab_size, embedding_dim)\n",
    "        \n",
    "        # CNN层用于局部特征提取\n",
    "        self.cnn_layers = nn.Sequential(\n",
    "            nn.Conv1d(embedding_dim, 256, kernel_size=3, padding=1),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool1d(kernel_size=2, stride=2),\n",
    "            nn.Conv1d(256, d_model, kernel_size=3, padding=1),\n",
    "            nn.ReLU(),\n",
    "            nn.AdaptiveAvgPool1d(max_seq_len // 2)  # 调整序列长度\n",
    "        )\n",
    "        \n",
    "        # 线性层调整维度\n",
    "        self.input_projection = nn.Linear(d_model, d_model)\n",
    "        \n",
    "        # 位置编码\n",
    "        self.pos_encoder = PositionalEncoding(d_model, max_seq_len)\n",
    "        \n",
    "        # Transformer编码器层\n",
    "        self.transformer_layers = nn.ModuleList([\n",
    "            TransformerEncoderLayer(d_model, nhead) \n",
    "            for _ in range(num_layers)\n",
    "        ])\n",
    "        \n",
    "        # 分类头\n",
    "        self.classifier = nn.Sequential(\n",
    "            nn.Linear(d_model, d_model // 2),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(0.5),\n",
    "            nn.Linear(d_model // 2, num_classes)\n",
    "        )\n",
    "        \n",
    "        self.d_model = d_model\n",
    "        self.max_seq_len = max_seq_len\n",
    "\n",
    "    def forward(self, x):\n",
    "        # x shape: (batch_size, seq_len)\n",
    "        \n",
    "        # 1. 词嵌入\n",
    "        embedded = self.embedding(x)  # (batch_size, seq_len, embedding_dim)\n",
    "        \n",
    "        # 2. 转换为CNN输入格式 (batch_size, embedding_dim, seq_len)\n",
    "        embedded = embedded.transpose(1, 2)\n",
    "        \n",
    "        # 3. 通过CNN层提取局部特征\n",
    "        cnn_output = self.cnn_layers(embedded)  # (batch_size, d_model, new_seq_len)\n",
    "        \n",
    "        # 4. 转换回Transformer格式 (batch_size, new_seq_len, d_model)\n",
    "        features = cnn_output.transpose(1, 2)\n",
    "        \n",
    "        # 5. 通过线性层调整维度\n",
    "        features = self.input_projection(features)\n",
    "        \n",
    "        # 6. 添加位置编码\n",
    "        features = self.pos_encoder(features)\n",
    "        \n",
    "        # 7. 通过Transformer编码器层\n",
    "        output = features\n",
    "        for layer in self.transformer_layers:\n",
    "            output = layer(output)\n",
    "        \n",
    "        # 8. 对序列维度进行平均池化\n",
    "        pooled_output = torch.mean(output, dim=1)  # (batch_size, d_model)\n",
    "        \n",
    "        # 9. 通过分类头得到最终预测\n",
    "        logits = self.classifier(pooled_output)  # (batch_size, num_classes)\n",
    "        \n",
    "        return logits\n",
    "\n",
    "# 模型使用示例\n",
    "def demo_model():\n",
    "    print(\"演示CNN-Transformer综合模型...\")\n",
    "    \n",
    "    # 图像分类模型示例\n",
    "    print(\"\\n1. 图像分类模型:\")\n",
    "    image_model = CNNTransformerModel(num_classes=10)\n",
    "    \n",
    "    # 模拟输入图像 (batch_size=4, channels=3, height=32, width=32)\n",
    "    sample_images = torch.randn(4, 3, 32, 32)\n",
    "    \n",
    "    with torch.no_grad():\n",
    "        image_outputs = image_model(sample_images)\n",
    "        print(f\"输入图像形状: {sample_images.shape}\")\n",
    "        print(f\"输出形状: {image_outputs.shape}\")\n",
    "    \n",
    "    # 序列分类模型示例\n",
    "    print(\"\\n2. 序列分类模型:\")\n",
    "    seq_model = SequenceCNNTransformerModel(vocab_size=10000, num_classes=2)\n",
    "    \n",
    "    # 模拟输入序列 (batch_size=4, seq_len=50)\n",
    "    sample_sequences = torch.randint(1, 10000, (4, 50))\n",
    "    \n",
    "    with torch.no_grad():\n",
    "        seq_outputs = seq_model(sample_sequences)\n",
    "        print(f\"输入序列形状: {sample_sequences.shape}\")\n",
    "        print(f\"输出形状: {seq_outputs.shape}\")\n",
    "    \n",
    "    return image_model, seq_model\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    # 检查设备\n",
    "    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "    print(f\"使用设备: {device}\")\n",
    "    \n",
    "    # 打印模型架构说明\n",
    "    print(\"CNN-Transformer综合模型架构说明:\")\n",
    "    print(\"- CNN部分: 提取局部空间/序列特征\")\n",
    "    print(\"- Transformer部分: 捕捉长距离依赖关系\")\n",
    "    print(\"- 两种架构结合: 充分利用CNN的局部特征提取能力和Transformer的全局建模能力\")\n",
    "    \n",
    "    # 创建模型并打印结构\n",
    "    print(\"\\n图像分类模型结构:\")\n",
    "    image_model = CNNTransformerModel(num_classes=10)\n",
    "    print(image_model)\n",
    "    \n",
    "    # 计算参数量\n",
    "    pytorch_total_params = sum(p.numel() for p in image_model.parameters())\n",
    "    print(f\"\\n图像分类模型总参数量: {pytorch_total_params}\")\n",
    "    \n",
    "    # 运行演示\n",
    "    demo_model()\n",
    "\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "cv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.10.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
